Head-mounted displays are widely used in gaming and training applications. Such head-mounted displays typically use electronically controlled displays mounted on a pair of glasses or a helmet with supporting structures such as ear, neck, or head pieces that are worn on a user's head. Displays are built into the glasses together with suitable optics to present electronic imagery to a user's eyes.
Most head-mounted displays provide an immersive effect in which scenes from the real world are obscured and the user can see, or is intended to see, only the imagery presented by the displays. In the present application, immersive displays are considered to be those displays that are intended to obscure a user's view of the real world to present information to the user from the display. Immersive displays can include cameras to capture images of the scene in front of the user so that this image information can be combined with other images to provide a combined image of the scene where portions of the scene image have been replaced to create a virtual image of the scene. In such an arrangement, the display area is opaque. Such displays are commercially available, for example from Vuzix.
United States Patent Application 2007/0237491 presents a head-mounted display that can be changed between an opaque mode where image information is presented and a see-through mode where the image information is not presented and the display is transparent. This mode change is accomplished by a manual switch that is operated by the user's hand or a face-muscle motion. This head-mounted display is either opaque or fully transparent.
Head-mounted displays can provide a see-through display for an augmented-reality view in which real-world scenes are visible to a user but additional image information is overlaid on the real-world scenes. Such an augmented-reality view is provided by helmet mounted displays found in military applications and by heads-up displays (HUDs) in the windshields of automobiles or aircraft. In this case, the display area is transparent. FIG. 10 shows a typical prior-art head-mounted display that is a see-through head-mounted display apparatus 10 in a glasses format. The head-mounted display apparatus 10 includes: ear pieces 14 to locate the device on the user's head; lens areas 12 that have variable occlusion members 7; microprojectors 8 and control electronics 9 to provide image information to at least the variable occlusion members 7.
U.S. Pat. No. 6,829,095 describes a device with a see-through head-mounted display apparatus 10 or augmented-reality display in a glasses format where image information is presented within the lens areas 12 of the glasses. The lens areas 12 of the glasses in this patent include waveguides to carry the image information to be displayed from an image source, with a built-in array of partially reflective surfaces to reflect the information out of the waveguide in the direction of the user's eyes. FIG. 11A shows a cross-section of a lens area 12 including: a waveguide 13; partial reflectors 3 along with; a microprojector 8 to supply a digital image; light rays 4 passing from the microprojector 8, through the waveguide 13, partially reflecting off the partial reflectors 3 and continuing on to the user's eye 2. As seen in FIG. 11A, light rays 5 from the ambient environment pass through the waveguide 13 and partial reflectors 3 as well as the transparent surrounding area of the lens area 12 to combine with the light 4 from the microprojector 8 and continue on to the user's eye 2 to form a combined image. The combined image in the area of the partial reflectors 3 is extra bright because light is received by the user's eye 2 from both the microprojector 8 and light rays 5 from the ambient environment.
U.S. Pat. No. 7,710,655 describes a variable occlusion member that is attached to a see-through display as a layer in an area in which image information is presented by the display. The layer of the variable occlusion member is used to limit the ambient light that passes through the see-through display from the external environment. The variable occlusion layer is adjusted from dark to light in response to the brightness of the ambient environment to maintain desirable viewing conditions. FIG. 10 shows a variable occlusion member 7 located in the center of the lens area 12 wherein the variable occlusion member 7 is in a transparent state so that scene light can pass through the variable occlusion member 7 to a viewer's eyes. FIG. 11A shows a variable occlusion member 7 wherein the variable occlusion member 7 is in a transparent state. In contrast, FIG. 11 B shows a cross-section of a variable occlusion member 7 in relation to the waveguide 13 and the partial reflectors 3 wherein the variable occlusion member 7 is in a darkened state so that light rays 5 from the ambient environment are substantially blocked in the area of the variable occlusion member 7 and light rays 5 from the ambient environment only pass through the transparent surrounding area of lens area 12 to continue on the user's eye 2. As a result, the combined image seen by the user is not overly bright in the area of the variable occlusion member 7 because substantially only light rays 4 passing from the microprojector 8 is seen in that area. FIG. 12 illustrates the variable occlusion member 7 in a dark state. Although image quality is improved by the method of U.S. Pat. No. 7,710,655, further improvements are needed to address motion sickness.
Motion sickness is a significant obstacle for users of immersive and virtual reality systems and head-mounted displays, limiting their widespread adoption despite their advantages in a range of applications in gaming and entertainment, military, education, medical therapy and augmented reality. Motion sickness or simulator sickness is a known problem for immersive displays because the user cannot see the environment well. As a result, motion on the part of a user, for example head motion, does not correspond to motion on the part of the display or imagery presented to the user by the display. This is particularly true for displayed video sequences that incorporate images of moving scenes that do not correspond to a user's physical motion. Motion-sickness symptoms are known to occur in users wearing head-mounted displays during head or body motion, as well as when watching content or playing computer games for a relatively prolonged period even without head or body motion.
“Motion sickness” is the general term describing a group of common symptoms such as nausea, vomiting, dizziness, vertigo, disorientation, sweating, fatigue, ataxia, fullness of stomach, pallor. Although sea-, car-, and airsickness are the most commonly experienced examples, these symptoms were discovered in other situations such as watching movies, video, in flight simulators, or in space. There are presently several conflicting theories trying to explain motion sickness and its variants. Three main theories are summarized below.
First, sensory conflict theory explains motion sickness symptoms as appearing when people are exposed to conditions of exogenous (non-volitional) motion and sensory rearrangement, when the rules which define the normal relationships between body movements and the resulting neural inflow to the central nervous system have been systematically changed. When the central nervous system receives sensory information concerning the orientation and movement of the body which is unexpected or unfamiliar in the context of motor intentions and previous sensory-motor experience and this condition persists for a relatively long time, motion sickness typically results. In the case of flight simulators and wide-screen movie theaters that create immersive visual experience, visual cues to motion are not matched by the usual pattern of vestibular and proprioceptive cues to body acceleration, which leads to motion sickness. Previous sensory motor experience also plays a role in the severity of the effects. Sensory conflict results from a mismatch between actual and anticipated sensory signals. In each specific experiential situation, different sensory signals can play a role and therefore different mitigation strategies are proposed, though vestibular, motor and visual systems are being recognized among the main sources for the sensory conflict.
Second, the poison theory attempts to explain motion-sickness phenomena from an evolutionary standpoint. It suggests that the ingestion of poison causes physiological effects involving the coordination of the visual, vestibular and other sensory input systems. They act as an early-warning system, which enhances survival by removing the contents of the stomach. Stimulation that is occurring in virtual and other environments, consequently associated with motion sickness provokes reaction of the visual and vestibular systems in such a way that it is misinterpreted by the body as resulting from the ingestion of some type of toxic substance and therefore causes motion sickness symptoms.
Third, the postural instability theory is based on the supposition that one of the primary behavioral goals in humans is to maintain postural stability in the environment, which is defined as the state with reduced uncontrolled movements of perception and action systems. This postural stability depends on the surrounding environment. If the environment changes abruptly or significantly, postural control will be lost or diminished, especially if a person's experience with such an environment is limited or lacking. In such a case, the person will be in a state of postural instability till the control strategy is learned and postural stability attained. Therefore, the postural instability theory states that the cause of motion sickness lies in prolonged postural instability, wherein the severity of the symptoms increases with the duration of the instability. A number of environmental situations can induce long periods of postural instability: low frequency vibration; weightlessness; changing relationships between the gravito-inertial force vector and the surface of support; and altered specificity, the factor that is relevant to motion sickness and other conditions when there is no obvious body motion. In these cases, visual characteristics (for example, visual scene motion, optical flow, or accelerations,) are unrelated to the constraints on control of body, therefore postural control strategies for gaining postural stability will not work. For example, a subject can use muscular force or even subtle movements to respond to visually perceived situations that do not correspond to the real physical environment, evoking thus a deviation from a stable position and causing postural instability. Other theories suggest eye movements or multi-factor explanations of motion sickness.
Motion sickness that occurs in the absence of body or head motion are of special interest and importance since head-mounted displays are becoming wide-spread for gaming applications, virtual-reality systems, and as personal viewing devices. Military flight simulators users develop signs and symptoms normally associated with classic motion sickness, such as nausea, pallor, sweating, or disorientation. In these cases, users have a compelling sense of self motion through moving visual imagery. This phenomenon has been referred to as “visually-induced motion sickness” to underscore its dependence on the visual stimulation in contrast to other symptoms collectively referred to as “asthenopia” and expressed as eyestrain, headache and blurred vision. Motion sickness depends at least in part on properties of visual stimulation. Visual scene motion and scene kinematics roughly corresponding to roll, pitch and flow present in the visual imagery correlates with sickness symptoms of nausea, disorientation and oculo-motor discomfort.
Visually induced motion sickness has been associated with the characteristics of visually presented content that are related to moving scenes, objects, optical flow, and other parameters indicating motion. It has also been reported that immersive environments with their large field-of-view displays, although enhancing a feeling of presence, can aggravate the symptoms. Summarily, these findings emphasize motion signals present in the scene as the factor behind the adverse effects. Such stimuli can elicit specific eye movement related reactions, namely, optokinetic nystagmus and the vestibulo-ocular reflex, which serve to provide image stabilization on the retina while observers maintain target fixation.
The illusion of self motion, referred to as vection, can lead to disorientation, one of the symptoms of motion sickness and has been cited as a key indicator of sickness symptoms in simulators and VR systems. In addition to low-level visual features influencing self-motion perception in virtual-reality systems, high-level cognitive factors have been shown to increase the illusion. The high degree of naturalism of the large distant visual surroundings in immersive environments that signifies “global scene consistency”, that is the coherence of a scene layout with our natural environment, led to the increased likelihood of inducing vection.
Several methods calculate parameters of moving objects and scenes to quantify aspects of visual imagery that can lead to motion-sickness-like symptoms. However, these tend to emphasize presence of motion in the scenery and derive measures of such motion. It is therefore difficult using such measures to predict whether images that do not have explicit motion signals will lead to motion sickness symptoms. Additionally, such measures do not take into consideration other visual signal modalities such as contrast, color (hue, saturation), luminance, depth, orientation which can increase adverse motion sickness related effects. Moreover, the existing methods cannot predict whether visual stimulation can provoke head and eye movements, which can cause motion-sickness-like symptoms in users of head-mounted displays in the absence of locomotion.
Despite voluminous research on motion sickness and related conditions, a solution to the problem has not yet been found, though several strategies have been suggested in the prior art to reduce the problem. For example, an invention disclosed in U.S. Pat. No. 6,497,649 by Parker et al describes a method for displaying an independent visual background including visual cues that are matched to the perception of motion by the vestibular system. The motion perceived by the vestibular system is detected by electromechanical sensors, the output of which is transformed through a perceptual model to produce perceptually relevant signals. The independent visual background is based upon these signals and is used to alleviate motion, simulator and virtual environment sickness. The method was designed primarily for the users of virtual reality and immersive systems and was shown to help when presented in the center of the visual field which essentially disrupts a viewing experience in a rather unnatural way. Similarly limited to a specific condition is an invention described in U.S. Pat. No. 7,128,705 by Brendley et al. disclosing a motion-coupled visual environment for the prevention or reduction of motion and simulator sickness to address the problems encountered by a user on a moving platform. The invention operates by sensing and signaling the inertial movement of the platform, displaying a window for the user to focus on, and moving the window in a way that correlates the perceived motion with the sensed inertial motion.
U.S. Pat. No. 6,497,649 discloses a method for reducing motion sickness produced by head movements when viewing a head-mounted immersive display. The patent describes the presentation of a texture field surrounding the displayed image information, wherein the texture field is moved in response to head movements of the user. This patent is directed at immersive displays.
The detrimental impact of motion sickness symptoms on the user and existing limitations of solutions on one hand, and desirability and potential utility of head-worn displays on the other hand, underscore the need to develop better methods to alleviate motion-sicknesses that take into consideration content information, external scene, or environment information and subjects' characteristics and which can operate even when no subject's body motion is expected.
Motion sickness is less of an issue for augmented-reality displays since the user can see the environment better, however, the imaging experience is not suitable for viewing high-quality images such as movies with a see-through display due to competing image information from the external scene overlaid with the image content and a resulting degradation in contrast and general image quality.
There is a need, therefore, for an improved head-mounted display system that enables viewing of high-quality image information with reduced motion sickness and improved viewing comfort for the user.